System and method for standardized evaluation of activity sequence

ABSTRACT

Motion windows are generated from a query activity sequence. For each of the motion windows in the query activity sequence, a corresponding motion window in the reference activity sequence is found. One or more difference calculations are performed between the motion windows of the query activity sequence and the corresponding motion windows in the reference activity sequence based on at least one criterion associated with physical meaning. Abnormality of the motion windows is determined based on the one or more difference calculations. A standardized evaluation result of the query activity sequence is output based on the detected abnormal motion windows in the query activity sequence.

TECHNICAL FIELD

The present disclosure relates to systems and methods for standardizedevaluation of activity sequences by using criteria associated withphysical meaning.

BACKGROUND

Aspects of human movement can have a large impact on, for example, theway a tool is designed, the way a workspace is laid out, or the way atask is performed. Understanding how the human body can move andinteract with objects and the environment can result in tools that aremore ergonomic, workspaces that are more efficient to navigate, andtasks that more intuitive to perform. The range of possible humanmotions and gestures is vast, however, and simple tasks, such as liftinga cup, pointing in a direction, or turning a screw, often result from acomplex set of biomechanical interactions. This relation of simpleresult from complex movement can make human motions and gesturesextremely difficult to quantify or understand in a meaningful orpractical way.

SUMMARY

In one or more illustrative examples, a system for detecting abnormalmotions in activity sequences includes a display device, a memoryconfigured to store a motion analysis application and motion capturedata including a reference activity sequence and a query activitysequence; and a processor, operatively connected to the memory and thedisplay device. The processor is configured to generate motion windowsfrom the query activity sequence; for each of the motion windows in thequery activity sequence, find a corresponding motion window in thereference activity sequence; perform one or more difference calculationsbetween the motion windows of the query activity sequence and thecorresponding motion windows in the reference activity sequence based onat least one criterion associated with physical meaning; determineabnormality of the motion windows according to the one or moredifference calculations; and output, to the display device, astandardized evaluation result of the query activity sequence indicativeof a measure of abnormality of the motion windows in the query activitysequence.

In one or more illustrative examples, a method for detecting abnormalmotions in activity sequences includes generating motion windows from aquery activity sequence; for each of the motion windows in the queryactivity sequence, finding a corresponding motion window in a referenceactivity sequence; performing one or more difference calculationsbetween the motion windows of the query activity sequence and thecorresponding motion windows in the reference activity sequence based onat least one criterion associated with physical meaning; determiningabnormality of the motion windows according to the one or moredifference calculations; and outputting a standardized evaluation resultof the query activity sequence indicative of a measure of abnormality ofthe motion windows in the query activity sequence.

In one or more illustrative examples, a non-transitory computer readablemedium includes instructions of a motion analysis application that, whenexecuted by one or more processors, cause the one or more processors togenerate motion windows from a query activity sequence; for each of themotion windows in the query activity sequence, find a correspondingmotion window in a reference activity sequence; perform a plurality ofdifference calculations between the motion windows of the query activitysequence and the corresponding motion windows in the reference activitysequence based on multiple criteria associated with physical meaning,the difference calculations including an orientation differencecalculation and a motion time difference calculation; perform a fusionof the plurality of difference calculations to determine abnormality ofthe motion windows; and output a standardized evaluation result of thequery activity sequence indicative of a measure of abnormality of themotion windows in the query activity sequence, the standardizedevaluation results including indications of motion windows identified ashaving abnormality.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example system for performing a standardizedevaluation of activity sequences according to this disclosure;

FIG. 2 illustrates an example process for evaluating a standardizationlevel of query activity sequence compared with a reference activitysequence;

FIG. 3 illustrates an example representation of a motion window for aquery activity sequence;

FIG. 4 illustrates an example calculation of the abnormality of motionwindows based on orientation difference; and

FIG. 5 illustrates an example calculation of the abnormality of motionwindow based on motion time difference.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described herein. It is to beunderstood, however, that the disclosed embodiments are merely examplesand other embodiments can take various and alternative forms. Thefigures are not necessarily to scale; some features could be exaggeratedor minimized to show details of particular components. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a representative basis forteaching one skilled in the art to variously employ the embodiments. Asthose of ordinary skill in the art will understand, various featuresillustrated and described with reference to any one of the figures canbe combined with features illustrated in one or more other figures toproduce embodiments that are not explicitly illustrated or described.The combinations of features illustrated provide representativeembodiments for typical applications. Various combinations andmodifications of the features consistent with the teachings of thisdisclosure, however, could be desired for particular applications orimplementations.

Understanding human activities based on inertial measurement units(IMUs) sensing data is an important, yet challenging, problem inindustry. As an essential topic towards activity understanding, in manyscenarios it is necessary to evaluate the standardization level of humanactivities. For example, inside manufactories, clearly understanding andevaluating the assembly actions of the operators can lead to improvedproduct quality management. However, manual inspection to perform suchevaluations is labor-intensive and it is thus necessary to develop asolution to evaluate the standardization level automatically. Somesystems may record a standardized operation beforehand and thencalculate a difference between each query activity sequence and thereference. Such approaches only suggest an abstract value as the overalldifference between the two activities. Furthermore, to give meaning tothe result, it is required for the value to be mapped to astandardization level based on human input each time according to therunning scenario.

This disclosure proposes an approach to evaluate the standardizationlevel of human physical activities using the time-series data from IMUs.The approach compares the query activity sequence with a given referenceactivity sequence, and fuses the evaluation results considering multiplecharacteristics of the physical motions. As compared to other systems,which evaluate activities as a whole and return a single number as thestandardization level, the proposed approach detects abnormal motions inthe activity sequence, in order to provide a local and fine-grainedevaluation result. Another significant advantage of the approach is thatthe thresholding variables may be associated with physical meaning andcan be generically defined for a wide range of activities, whereas inprevious approaches the parameters are manually selected for each case.

FIG. 1 is a schematic diagram of an exemplary embodiment of a system 100for performing a standardized evaluation of activity sequences. Thesystem 100 may quantitatively compute an accuracy, e.g., a deviationfrom a prescribed motion or gesture and deviation from a target timeperiod for completion, of a movement. The system 100 includes aprocessor 102 that is operatively connected to a memory 110, inputdevice 118, motion capture device 120, and a display device 108. As isdescribed in more detail below, during operation, the system 100 (i)receives motion capture data 114 including a reference activity sequenceand a query activity sequence, e.g., from the memory 110, input device118, or another source, (ii) processes the query activity sequence permotion window to compare to the reference activity sequence, (iii)performs a plurality of analyses and a data fusion of the analyses todetermine abnormality of the motion window, and (iv) generatesstandardized evaluation results 116 of the computation that identifiesabnormal motion windows and a standardized evaluation of the queryactivity sequence.

In the system 100, the processor 102 includes one or more integratedcircuits that implement the functionality of a central processing unit(CPU) 104 and graphics processing unit (GPU) 106. In some examples, theprocessor 102 is a system on a chip (SoC) that integrates thefunctionality of the CPU 104 and GPU 106, and optionally othercomponents including, for example, the memory 110, a network device, anda positioning system, into a single integrated device. In other examplesthe CPU 104 and GPU 106 are connected to each other via a peripheralconnection device such as PCI express or another suitable peripheraldata connection. In one example, the CPU 104 is a commercially availablecentral processing device that implements an instruction set such as oneof the x86, ARM, Power, or MIPS instruction set families.

The GPU 106 may include hardware and software for display of at leasttwo-dimensional (2D) and optionally three-dimensional (3D) graphics to adisplay device 108. The display device 108 may include an electronicdisplay screen, projector, printer, or any other suitable device thatreproduces a graphical display. In some examples, processor 102 executessoftware programs including drivers and other software instructionsusing the hardware functionality in the GPU 106 to accelerate generationand display of the graphical depictions of models of human movement andvisualizations of quantitative computations that are described herein

During operation, the CPU 104 and GPU 106 execute stored programinstructions that are retrieved from the memory 110. The stored programinstructions include software that control the operation of the CPU 104and the GPU 106 to perform the operations described herein.

While FIG. 1 depicts the processor 102 as including both the CPU 104 andGPU 106, alternative embodiments may omit the GPU 106, as for examplethe processor 102 may be of a server that generates output visualizationdata using only a CPU 104 and transmits the output visualization data toa remote client computing device that uses a GPU 106 and a displaydevice 108 to display the data. Additionally, alternative embodiments ofthe processor 102 can include microcontrollers, application specificintegrated circuits (ASICs), field programmable gate arrays (FPGAs),digital signal processors (DSPs), or any other suitable digital logicdevices in addition to or as replacements of the CPU 104 and GPU 106.

In the system 100, the memory 110 includes both non-volatile memory andvolatile memory devices. The non-volatile memory includes solid-statememories, such as NAND flash memory, magnetic and optical storage media,or any other suitable data storage device that retains data when thesystem 100 is deactivated or loses electrical power. The volatile memoryincludes static and dynamic random-access memory (RAM) that storesprogram instructions and data, including a motion analysis application112, motion capture data 114, and standardized evaluation results 116,during operation of the system 100. In some embodiments the CPU 104 andthe GPU 106 each have access to separate RAM devices (e.g., a variant ofDDR SDRAM for the CPU 104 and a variant of GDDR, HBM, or other RAM forthe GPU 106) while in other embodiments the CPU 104 and GPU 106 access ashared memory device. The memory 110 may store the motion capture data114, motion analysis application 112, and standardized evaluationresults 116 for maintenance and retrieval.

The input device 118 may include any of various devices that enable thesystem 100 to receive the motion capture data 114, motion analysisapplication 112, and standardized evaluation results 116. Examples ofsuitable input devices include human interface inputs such as keyboards,mice, touchscreens, voice input devices, and the like, as well. In someexamples the system 100 implements the input device 118 as a networkadapter or peripheral interconnection device that receives data fromanother computer or external data storage device, which can be usefulfor receiving large sets of motion capture data 114 in an efficientmanner.

The display device 108 may include an electronic display screen,projector, printer, or any other suitable device that reproduces agraphical display of the standardized evaluation results 116 that thesystem 100 generates based on the motion capture data 114.

The motion analysis application 112 includes instructions that, whenexecuted by the processor 102 of the system 100, cause the system 100 toperform the processes and operations described herein. These processesand operations include to receive motion capture data 114 including areference activity sequence and a query activity sequence, e.g., fromthe memory 110, input device 118, or another source, (ii) process thequery activity sequence per motion window to compare to the referenceactivity sequence, (iii) perform a plurality of analyses and a datafusion of the analyses to determine abnormality of the motion window,and (iv) generate standardized evaluation results 116 of the computationthat identifies abnormal motion windows and a standardized evaluation ofthe query activity sequence.

The motion capture data 114 refers to a plurality of recordsrepresentative of the locations of at least one tracked item or portionof the item over time. For example, the motion capture data 114 mayinclude one or more of: records of positions of a reference point on abody part over time or at set time intervals, sensor data taken overtime, a video stream or a video stream that has been processed using acomputer-vision technique, data indicative of the operating state of amachine over time, etc. In some cases, the motion capture data 114 mayinclude data representative of more than one continuous movement. Forinstance, the motion capture data 114 may include a combination of aplurality of combined motion capture data 114 sets.

A motion capture device 120 is a device configured to generate motioncapture data 114. Motion capture devices 120 may include, as somenon-limiting examples: cameras, visual sensors, infra-red sensors,ultrasonic sensors, accelerometers, pressure sensors, or the like. Onenon-limiting example of a motion capture device 120 is one or a pair ofdigital gloves that a user wears while performing cyclical motions. Thedigital gloves may include sensors that capture the motions of the userto generate the motion capture data 114 that are stored in the memory110.

A movement is an action performed by an operator. A reference movementrefers to a baseline or canonical version of the movement. The referencemovement may be used as a standard of comparison for other movements, toallow for identification of how close the other movements are to thereference movement.

The motion capture data 114 may be generally classified into one of twocategories for the purpose of computing the accuracy of a human motion:a reference activity sequence 122 that includes data representative ofthe reference or baseline movement, and a query activity sequence 124that includes data representative of the test movement, i.e., a movementto be compared and quantitatively evaluated for accuracy relative to thebaseline movement.

The reference activity sequence 122 may include motion capture data 114received from the motion capture device 120. The data may also includeprocessed movement data, such as, for example, frame, step, cycle, andtime information gleaned from the raw movement data. In one example, thereference movement is represented as a reference activity sequence 122having an ordered set of frames that each includes motion capture data114 corresponding to a respective interval of time of the referencemovement.

The query activity sequence 124 may also include motion capture data 114received from the motion capture device 120. In some examples, amovement or movements in the motion capture data 114 includes a label orlabels classifying the movements as either reference movements or testmovements. The motion analysis application 112 may be programmed isconfigured to receive instruction for classifying a movement ormovements as reference movements or test movements, such as from a uservia the input device 118 or from another source.

The motion analysis application 112 may be programmed to separate motioncapture data 114 into individual movements. In some examples, the motionanalysis application 112 computes different possible separations of themotion capture data 114 into individual movements and selects aseparation based on accuracies computed by the motion analysisapplication 112.

The motion analysis application 112 may also be programmed to separate areceived movement into frames, whereby a “frame” corresponds to adiscrete interval of time. In other words, each frame of a movementincludes a portion of the motion capture data 114 corresponding to aportion of the movement occurring during a respective interval of thetimeline for that movement. In some examples, the duration for theinterval corresponding to an individual frame is preset. In someexamples, the duration for the interval corresponding to an individualframe is set based on an instruction received from, for example, theuser via the input device 118 or another source. In some examples, theduration for the interval corresponding to an individual frame is setwith reference to one or more characteristics of the motion capture data114. For example, in some embodiments, the duration for the intervalcorresponding to an individual frame is set with reference to one ormore of a duration of a reference movement, a total travel distance forthe movement, a number of individual motions or gestures within themovement, a speed of the movement, etc. Generally, the same interval forthe duration of frames is used for both a reference movement and fortest movements to be evaluated relative to the reference movement.

In some instances, motion capture data 114 may be received as a file ofstored motion capture data from a data storage device. In suchinstances, the motion analysis application 112 may separate the movementor movements in the motion capture data 114 into frames for furtherprocessing.

The motion analysis application 112 may also map frames of the testmovement to corresponding frames of the reference movement. In someexamples, the test movement and reference movement are synchronized sothat frames of the test movement are mapped to frames of the referencemovement that correspond temporally, and in some embodiments, the testmovement and the reference movement are aligned in terms of gestures andmotions within the movement, such that frames of the test movement aremapped to frames of the reference movement that correspond with regardto the sequence of motions and/or gestures performed in the movement.

The motion analysis application 112 may further compute an accuracy ofthe test movement or test movements relative to the reference movement.Based on the analysis, the motion analysis application 112 may generatethe standardized evaluation results 116. The standardized evaluationresults 116 may include a standardized level score indicative of anobjective amount of difference between the test movements and thereference movement. The standardized evaluation results 116 may furtherinclude information indicative of the abnormal actions within the testmovements as compared to the reference movement. Further aspects ofgeneration of the standardized evaluation results 116 are described indetail below.

While the illustrated system 100 is shown using a single computingdevice that incorporates the display device 108, other example systems100 may include multiple computing devices. As one example, theprocessor 102 generates the standardized evaluation results 116 as oneor more data files, and the processor 102 transmits the standardizedevaluation results 116 to a remote computing device via a data network.The remote computing device then may display the output standardizedevaluation results 116. In one nonlimiting example, the processor 102 isimplemented in a server computing device that executes the motionanalysis application 112 to implement a web server that transmits thestandardized evaluation results 116 to a web browser in a remote clientcomputing device via a data network. The client computing deviceimplements a web browser or other suitable image display software todisplay the standardized evaluation results 116 received from the serverusing a display device 108 of the client computing device.

FIG. 2 illustrates an example process 200 for evaluating astandardization level of query activity sequence 124 compared with areference activity sequence 122. As used herein, a motion window is atime window with a motion defined in that time window. The motionwindows may be generated from the query activity sequence 124. Then, foreach iteration, the motion window in query activity sequence 124 findsthe corresponding motion window in reference activity sequence 122. Withthe motion windows identified, a difference between the two motionwindows may be calculated based on one or more criteria. In oneillustrative implementation, the difference between two motion windowsis calculated based on two criteria, namely orientation and motion time.The final abnormality of the motion window may be determined by fusingthe results from the criteria. The iteration continues until all motionwindows are processed. The standardization evaluation of the queryactivity sequence is calculated based on the detected abnormal motionwindows. The process 200 may be performed by the processor 102 of thesystem 100, as discussed in more detail below.

More specifically, at operation 202 the processor 102 receives the queryactivity sequence 124 and the reference activity sequence 122. In someexamples, motion capture data 114 of the query activity sequence 124 andthe reference activity sequence 122 may be received from a motioncapture device 120. For instance, the motion capture device 120 may be aset of gloves (not shown) that, when worn by a user, is configured totransmit motion capture data 114 representative of the orientation ofthe user hands given by a palm-facing direction of each hand, a gesturefor each hand given by joint angles for the joints of each hand, and amovement given by the linear acceleration in three dimensions for eachhand. In some examples, the motion capture device 120 may be configuredto connect with the system 100 via a wireless connection protocol suchas, for example, BLUETOOTH, Wi-Fi, radio transmission, etc. In someexamples, the motion capture device 120 may include tracking pointswhich are trackable using an optical tracking system such as a camera orinfra-red tracking system. In some examples, the motion capture device120 may include one or more controls such as a button or switchconfigured to one or more of cause the system 100 to begin capturingmotion capture data 114 transmitted by the motion capture device 120 andcause the system 100 to cease capturing the motion capture data 114. Insome examples, the processor 102 may be configured to store the capturedmotion capture data 114 in a data file on a data storage device, such asin response to an instruction from the user received via the inputdevice 118.

In some examples, the motion capture data 114 may be received as a datafile from a data storage device. For instance, the processor 102 may beconfigured to receive user instruction via the input device 118 to loadmotion capture data 114 from a data file on a data storage device. Insome examples, portions of motion capture data 114 are received fromdifferent sources. For instance, the reference activity sequence 122 maybe loaded from a data file on a data storage device such as the memory110, while the query activity sequence 124 may be captured using themotion capture device 120.

Regardless of source, the input to the process 200 is the query activitysequence 124 and the reference activity sequence 122. In one or moreimplementations, the data in the query activity sequence 124 containsorientation data from the IMU sensors. A quaternion is used to representthe orientation. Accordingly, the activity sequence 124 includes of asequence of quaternions can be represented as shown in Equation (1):

q _(t) =w+ix+jy+kz  (1)

where t denotes time. The quaternion data representing the orientationinformation is the basis to determine the dissimilarity of the queryactivity sequence 124 compared to the reference activity sequence 122.

The processor 102 generates motion windows from the query activitysequence 124 at operation 204. FIG. 3 illustrates an example 300representation of a motion window for a query activity sequence 124. Themotion window may be referred to as w_(q) ^((i)), denoting the windowcentered at index i of the query activity sequence. For each frame ofthe query activity sequence 124, the processor 102 generates a motionwindow from the time window centered at that frame with a durationconfigured by a parameter (e.g., an amount of time, a number of frames,etc.). The motion window may be clamped at the border of the queryactivity sequence 124 for frames at indexes that lack a duration in bothdirections of the extent of the configured parameter. Generally, if aquery activity sequence 124 contains n frames, the processor 102generates n motion windows from that query activity sequence 124.

As shown at operation 206, the processor 102 finds corresponding motionwindows in the query activity sequence 124 and the reference activitysequence 122. Due to the high level of flexibility in movements of thequery activity sequence 124 and the reference activity sequence 122, thequery activity sequence 124 and the reference activity sequence 122 maybe synchronized such that the frame-level matching can be achieved.Various algorithms may be used for this task, including Dynamic TimeWarping (DTW) which is a process by which a warping path is determinedto align desynchronized portions of the query activity sequence 124 andthe reference activity sequence 122. The matching algorithm is usuallybased on the pairwise distance between sequence frames, in thisparticular case, quaternions. In an example, the following quaterniondistance d(q₁, q₂) is used, as shown in Equation (2):

d(q ₁ ,q ₂)=2 cos⁻¹(|

q ₁ ,q ₂

|)  (2)

where

q₁, q₂

denotes the quaternion inner product.

Given the set of pairwise distances, the processor 102 then identifiesthe optimal matching such that the overall distance is minimized. As aresult, the processor 102 obtains the matching between a frame in thequery activity sequence 124 and a frame in the reference activitysequence 122. Each frame of the motion window in query activity sequence124 accordingly is associated with its corresponding frames in referenceactivity sequence 122. This set of corresponding frames defines thecorresponding motion window in reference activity sequence 122, denotedas w_(r) ^((f)).

Regarding the difference calculation, the difference between the twomotion windows may be calculated based on one or more criteria. In oneimplementation, the difference between two motion windows is calculatedbased on two criteria, namely orientation and motion time.

As shown at operation 208, the processor 102 computes the orientationdifference. FIG. 4 illustrates an example 400 calculation of theabnormality of motion windows based on orientation difference. Theorientation difference of two frames can be defined using the quaterniondistance d(q₁, q₂) defined above, where q₁, q₂ are the quaternion valuescontained in the frame data. Similarly, the orientation differencebetween two motion windows can be calculated based on the differencebetween all frames in the two motion windows. However, the movements canoften exhibit large orientation variation, even for the same motion. Toaccommodate such variability, the median value of the pairwise distanceis selected to represent the distance of the two motion windows. Themotion window w_(q) ^((i)) from query activity sequence is determined tobe abnormal based on orientation difference if the median value is abovea predefined threshold value.

As shown at operation 210, the processor 102 computes the motion timedifference. FIG. 5 illustrates an example 500 calculation of theabnormality of motion window based on motion time difference. Theduration of a motion window is another strong indicator of thestandardization level. This invention considers the motion timedifference of the two motion windows. The length of motion window isdefined as the number of frames in the motion window. The motion windowis considered abnormal if the length difference between the two motionwindows, namely the one in query activity sequence and the counterpartin reference activity sequence, is larger than a certain threshold.

Fusion of the differences to determine abnormality of the motion windowis performed at operation 212. In an example, the abnormality of themotion window is determined by fusing the results from multiplecriteria. In some implementations, the difference between two motionwindows is calculated based on two criteria, namely orientation andmotion time continuing with the illustrated example. As one possibility,the motion window is identified as abnormal if any of the criterion issatisfied.

At operation 214, the processor 102 determines whether any additionalmotion windows require processing. If so, control returns to operation206. If all motion windows have been processed, control passes tooperation 216.

At 216, once all motion windows in the query activity sequence areevaluated, the processor 102 calculates the standardization evaluationresults 116 of the sequence based on the detected abnormal motionwindows in the sequence. One method is to give a standardization levelscore of the query activity sequence 124 by calculating the ratio ofabnormal motion windows to total number of motion windows. The score maybe calculated as shown in Equation (3):

$\begin{matrix}{{s = {1.0 - \frac{\sum_{i = 1}^{N_{q}}{{abnormal}\left( w_{q}^{(i)} \right)}}{N_{q}}}}{{where}\text{:}}{{{abnormal}\left( w_{q}^{(i)} \right)} = \left\{ {\begin{matrix}1 & {{if}\mspace{14mu} w_{q}^{(i)}\mspace{14mu} {is}\mspace{14mu} {abnormal}} \\0 & {otherwise}\end{matrix},{and}} \right.}} & (3)\end{matrix}$

N_(q) is the number of motion window in query activity sequence 124.

Accordingly, the disclosed approaches provide a solution to evaluate thestandardization level of a query activity sequence 124 automatically bydetecting abnormal motions inside the query activity sequence 124. Thisapproach may be useful for any system that leverages IMU for humanactivity (including, but not limited to particular hand motions) anomalydetection and standardization evaluation.

The processes, methods, or algorithms disclosed herein can bedeliverable to/implemented by a processing device, controller, orcomputer, which can include any existing programmable electronic controlunit or dedicated electronic control unit. Similarly, the processes,methods, or algorithms can be stored as data and instructions executableby a controller or computer in many forms including, but not limited to,information permanently stored on non-writable storage media such as ROMdevices and information alterably stored on writeable storage media suchas floppy disks, magnetic tapes, CDs, RAM devices, and other magneticand optical media. The processes, methods, or algorithms can also beimplemented in a software executable object. Alternatively, theprocesses, methods, or algorithms can be embodied in whole or in partusing suitable hardware components, such as Application SpecificIntegrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs),state machines, controllers or other hardware components or devices, ora combination of hardware, software and firmware components.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms encompassed by the claims.The words used in the specification are words of description rather thanlimitation, and it is understood that various changes can be madewithout departing from the spirit and scope of the disclosure. Aspreviously described, the features of various embodiments can becombined to form further embodiments of the invention that may not beexplicitly described or illustrated. While various embodiments couldhave been described as providing advantages or being preferred overother embodiments or prior art implementations with respect to one ormore desired characteristics, those of ordinary skill in the artrecognize that one or more features or characteristics can becompromised to achieve desired overall system attributes, which dependon the specific application and implementation. These attributes caninclude, but are not limited to cost, strength, durability, life cyclecost, marketability, appearance, packaging, size, serviceability,weight, manufacturability, ease of assembly, etc. As such, to the extentany embodiments are described as less desirable than other embodimentsor prior art implementations with respect to one or morecharacteristics, these embodiments are not outside the scope of thedisclosure and can be desirable for particular applications.

What is claimed is:
 1. A system for detecting abnormal motions inactivity sequences, comprising: a display device; a memory configured tostore a motion analysis application and motion capture data including areference activity sequence and a query activity sequence; and aprocessor, operatively connected to the memory and the display device,and configured to execute the motion analysis application to generatemotion windows from the query activity sequence, for each of the motionwindows in the query activity sequence, find a corresponding motionwindow in the reference activity sequence, perform one or moredifference calculations between the motion windows of the query activitysequence and the corresponding motion windows in the reference activitysequence based on at least one criterion associated with physicalmeaning, determine abnormality of the motion windows according to theone or more difference calculations, and output, to the display device,a standardized evaluation result of the query activity sequenceindicative of a measure of abnormality of the motion windows in thequery activity sequence.
 2. The system of claim 1, wherein the one ormore difference calculations includes an orientation differencecalculation and a motion time difference calculation, and to determineabnormality of the motion windows according to the one or moredifference calculations includes performing a fusion of results of theorientation difference calculation and the motion time differencecalculation.
 3. The system of claim 1, wherein the standardizedevaluation result indicates an abnormality for a motion window if anycriterion of the one or more difference calculations is satisfied. 4.The system of claim 1, wherein the processor is further configured toexecute the motion analysis application to output, to the displaydevice, any motion windows identified as having abnormality.
 5. Thesystem of claim 1, wherein the processor is further configured toexecute the motion analysis application to: calculate a ratio ofabnormal motion windows to total number of motion windows; and includethe ratio in the standardized evaluation result.
 6. The system of claim1, wherein the reference activity sequence and the query activitysequence include quaternion values that represent orientationinformation, and the processor is further configured to execute themotion analysis application to obtain orientation differences betweenthe motion windows of the reference activity sequence and the queryactivity sequence according to quaternion distance; and identify motionwindows of the query activity sequence as having abnormality accordingto the orientation differences exceeding a predefined threshold value.7. The system of claim 1, wherein the processor is further configured toexecute the motion analysis application to: identify motion timedifferences according to differences in length between motion windows ofthe reference activity sequence and the query activity sequence; andidentify motion windows of the query activity sequence as havingabnormality according to the differences in length exceeding apredefined threshold value.
 8. The system of claim 1, wherein the queryactivity sequence includes a plurality of frames, and the processor isfurther configured to execute the motion analysis application to, foreach frame of the query activity sequence, generate a motion window as atime window centered at that frame with a duration of a predefinedparameter.
 9. A method for detecting abnormal motions in activitysequences comprising: generating motion windows from a query activitysequence; for each of the motion windows in the query activity sequence,finding a corresponding motion window in a reference activity sequence;performing one or more difference calculations between the motionwindows of the query activity sequence and the corresponding motionwindows in the reference activity sequence based on at least onecriterion associated with physical meaning; determining abnormality ofthe motion windows according to the one or more difference calculations;and outputting a standardized evaluation result of the query activitysequence indicative of a measure of abnormality of the motion windows inthe query activity sequence.
 10. The method of claim 9, wherein the oneor more difference calculations includes an orientation differencecalculation and a motion time difference calculation, and furthercomprising determining abnormality of the motion windows according tothe one or more difference calculations by performing a fusion ofresults of the orientation difference calculation and the motion timedifference calculation.
 11. The method of claim 9, wherein thestandardized evaluation result indicates an abnormality for a motionwindow if any criterion of the one or more difference calculations issatisfied.
 12. The method of claim 9, further comprising outputting, toa display device, any motion windows identified as having abnormality.13. The method of claim 9, further comprising: calculating a ratio ofabnormal motion windows to total number of motion windows; and includingthe ratio in the standardized evaluation result.
 14. The method of claim9, wherein the reference activity sequence and the query activitysequence include quaternion values that represent orientationinformation, further comprising: obtaining orientation differencesbetween the motion windows of the reference activity sequence and thequery activity sequence according to quaternion distance; andidentifying motion windows of the query activity sequence as havingabnormality according to the orientation differences exceeding apredefined threshold value.
 15. The method of claim 9, furthercomprising: identifying motion time differences according to differencesin length between motion windows of the reference activity sequence andthe query activity sequence; and identifying motion windows of the queryactivity sequence as having abnormality according to the differences inlength exceeding a predefined threshold value.
 16. The method of claim9, wherein the query activity sequence includes a plurality of frames,and further comprising, for each frame of the query activity sequence,generating a motion window as a time window centered at that frame witha duration of a predefined parameter.
 17. A non-transitory computerreadable medium comprising instructions of a motion analysis applicationthat, when executed by one or more processors, cause the one or moreprocessors to: generate motion windows from a query activity sequence;for each of the motion windows in the query activity sequence, find acorresponding motion window in a reference activity sequence; perform aplurality of difference calculations between the motion windows of thequery activity sequence and the corresponding motion windows in thereference activity sequence based on multiple criteria associated withphysical meaning, the difference calculations including an orientationdifference calculation and a motion difference calculation; perform afusion of the plurality of difference calculations to determineabnormality of the motion windows; and output a standardized evaluationresult of the query activity sequence indicative of a measure ofabnormality of the motion windows in the query activity sequence, thestandardized evaluation result including indications of motion windowsidentified as having abnormality.
 18. The medium of claim 17, furthercomprising instructions of the motion analysis application that, whenexecuted by one or more processors, cause the one or more processors to:calculate a ratio of abnormal motion windows to total number of motionwindows; and include the ratio in the standardized evaluation result.19. The medium of claim 17, wherein the reference activity sequence andthe query activity sequence include quaternion values that representorientation information, and further comprising instructions of themotion analysis application that, when executed by one or moreprocessors, cause the one or more processors to: obtain orientationdifferences between the motion windows of the reference activitysequence and the query activity sequence according to quaterniondistance; and identify motion windows of the query activity sequence ashaving abnormality according to the orientation differences exceeding apredefined threshold value.
 20. The medium of claim 17, furthercomprising instructions of the motion analysis application that, whenexecuted by one or more processors, cause the one or more processors to:identify motion time differences according to differences in lengthbetween motion windows of the reference activity sequence and the queryactivity sequence; and identify motion windows of the query activitysequence as having abnormality according to the differences in lengthexceeding a predefined threshold value.